ABSTRACT
Purposes: Artificial Intelligence (AI) models are constantly developing to help clinicians in challenging tasks such as classification of images in radiological practice. The aim of this work was to compare the diagnostic performance of an AI classifier model developed in our hospital with the results obtained from the radiologists reading the CT images in discriminating different types of viral pneumonia. Material(s) and Method(s): Chest CT images of 1028 patients with positive swab for SARS-CoV-2 (n=646) and other respiratory viruses (n=382) were segmented automatically for lung extraction and Radiomic Features (RF) of first (n=18) and second (n=120) order were extracted using PyRadiomics tools. RF, together with patient age and sex, were used to develop a Multi-Layer Perceptron classifier to discriminate images of patients with COVID-19 and non-COVID-19 viral pneumonia. The model was trained with 808 CT images performing a LASSO regression (Least Absolute Shrinkage and Selection Operator), a hyper-parameter tuning and a final 4-fold cross validation. The remaining 220 CT images (n=151 COVID-19, n=69 non-COVID-19) were used as independent validation (IV) dataset. Four readers (three radiologists with >10 years of experience and one radiology resident with 3 years of experience) were recruited to blindly evaluate the IV dataset using the 5-points scale CO-RADS score. CT images with CO-RADS >=3 were considered "COVID-19". The same images were classified as "COVID-19" or "non-COVID-19" by applying the AI model with a threshold on the predicted values of 0.5. Diagnostic accuracy, specificity, sensibility and F1 score were calculated for human readers and AI model. Result(s): The AI model was trained using 24 relevant features while the Area under ROC curve values after 4-fold cross validation and its application to the IV dataset were, respectively, 0.89 and 0.85. Interreader agreement in assigning CO-RADS class, analyzed with Fleiss' kappa with ordinal weighting, was good (k=0.68;IC95% 0.63-0.72) and diagnostic performance were then averaged among readers. Diagnostic accuracy, specificity, sensibility and F1 score resulted 78.6%, 78.3%, 78.8% and 78.5% for AI model and 77.7%, 65.6%, 83.3% and 72.0% for human readers. The difference between specificity and sensitivity observed in human readers could be related to the higher rate of false positive due to the higher incidence of COVID-19 patients in comparison with other types of viral pneumonitis during the last 2 years. Conclusion(s): A model based on RF and artificial intelligence provides comparable results with human readers in terms of diagnostic performance in a classification task.Copyright © 2023 Southern Society for Clinical Investigation.
ABSTRACT
PURPOSE: To assess the impact of lung segmentation accuracy in an automatic pipeline for quantitative analysis of CT images. METHODS: Four different platforms for automatic lung segmentation based on convolutional neural network (CNN), region-growing technique and atlas-based algorithm were considered. The platforms were tested using CT images of 55 COVID-19 patients with severe lung impairment. Four radiologists assessed the segmentations using a 5-point qualitative score (QS). For each CT series, a manually revised reference segmentation (RS) was obtained. Histogram-based quantitative metrics (QM) were calculated from CT histogram using lung segmentationsfrom all platforms and RS. Dice index (DI) and differences of QMs (ΔQMs) were calculated between RS and other segmentations. RESULTS: Highest QS and lower ΔQMs values were associated to the CNN algorithm. However, only 45% CNN segmentations were judged to need no or only minimal corrections, and in only 17 cases (31%), automatic segmentations provided RS without manual corrections. Median values of the DI for the four algorithms ranged from 0.993 to 0.904. Significant differences for all QMs calculated between automatic segmentations and RS were found both when data were pooled together and stratified according to QS, indicating a relationship between qualitative and quantitative measurements. The most unstable QM was the histogram 90th percentile, with median ΔQMs values ranging from 10HU and 158HU between different algorithms. CONCLUSIONS: None of tested algorithms provided fully reliable segmentation. Segmentation accuracy impacts differently on different quantitative metrics, and each of them should be individually evaluated according to the purpose of subsequent analyses.
Subject(s)
COVID-19 , Algorithms , Humans , Image Processing, Computer-Assisted , Lung , Neural Networks, Computer , SARS-CoV-2 , Tomography, X-Ray ComputedABSTRACT
PURPOSE: Quantitative metrics in lung computed tomography (CT) images have been widely used, often without a clear connection with physiology. This work proposes a patient-independent model for the estimation of well-aerated volume of lungs in CT images (WAVE). METHODS: A Gaussian fit, with mean (Mu.f) and width (Sigma.f) values, was applied to the lower CT histogram data points of the lung to provide the estimation of the well-aerated lung volume (WAVE.f). Independence from CT reconstruction parameters and respiratory cycle was analysed using healthy lung CT images and 4DCT acquisitions. The Gaussian metrics and first order radiomic features calculated for a third cohort of COVID-19 patients were compared with those relative to healthy lungs. Each lung was further segmented in 24 subregions and a new biomarker derived from Gaussian fit parameter Mu.f was proposed to represent the local density changes. RESULTS: WAVE.f resulted independent from the respiratory motion in 80% of the cases. Differences of 1%, 2% and up to 14% resulted comparing a moderate iterative strength and FBP algorithm, 1 and 3 mm of slice thickness and different reconstruction kernel. Healthy subjects were significantly different from COVID-19 patients for all the metrics calculated. Graphical representation of the local biomarker provides spatial and quantitative information in a single 2D picture. CONCLUSIONS: Unlike other metrics based on fixed histogram thresholds, this model is able to consider the inter- and intra-subject variability. In addition, it defines a local biomarker to quantify the severity of the disease, independently of the observer.